Linear and Nonlinear Modeling of Item Position Effects

Item parameter invariance is one of the properties of item response theory (IRT) that enables computerized adaptive testing (CAT) for test administration. The possible influence of item position on test performance is one of the severe threats to the property of item parameter invariance within IRT. This study examines how different impact how the relationship between item position and item difficulty is expressed. An explanatory IRT model is formulated for estimating item position effects. The model is demonstrated using data from the Program for International Student Assessment (PISA) 2009 reading data for the U.S., wherein the same items appeared in four different positions across item clusters. Methods of choosing the best model to detect item position effects are discussed as well as preliminary item analysis for the estimation of item position effects. Item parameter invariance is one of the properties of item response theory (IRT) that enables computerized adaptive testing (CAT) for test administration (Meyers et al., 2009). The influence of item position on test performance is one of the severe threats to the property of item parameter invariance within item response theory (Leary & Dorans, 1985). In an era of rapid proliferation of CAT, item position effects on item difficulty and test performance is an important issue to examine. Since the position effects may cause unintended impact on test performance, test developers do not want the item position effects to be present in their tests. In that sense, item position effects on test performance could negatively impact the validity of test score interpretations and uses. Since the early work of Mollenkopf (1950), the unintended effects of item order on test performance have been studied thoroughly. Early studies of item order or context effects were mainly focused on their impact on item difficulty. These studies are roughly summarized as two types of effects: a fatigue effect and a learning effect. Fatigue effects occur when items become more difficult as they are located later in the test. Even though there could be many reasons for the decrease of the test performance, such as lower motivation or simply running out of time, studies tend to refer to decreases in